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SUMMARY 


\ 

An  F-16  advanced  electro-optical  (EO)  pod  field  of 
view  (FOV)  simulation  study  was  accomplished  to  determine 
the  effects  of  sensor  FOV  and  field-of -regard  (FOR)  on  the 
pilot's  workload  and  ability  to  locate,  acquire,  and  attack 
targets  in  a day  weapons-delivery  environment.  The  simula- 
ted EO  pod  had  a slewable  TV  sensor  and  laser  designator 
head  with  two  selectable  FOVs  for  the  TV  sensor  (wide  and 
narrow  FOVs) , selectable  area  correlation  and  contrast 
tracking  modes,  and  the  capability  to  be  either  slaved  to 
a preprogrammed  set  of  coordinate  or  slaved  to  the  air- 
craft's velocity  vector.  -Three  combinations  of  wide  and 
narrow  FOV  pairs  representative  of  current  EO  pod  designs 
were  examined  - 6 and  1.5  degrees,  6 and  0.86  degrees,  and 
3 and  0.86  degrees.  The  three  FOV  pairs  were  evaluated 
subjectively  and  objectively  in  three  experiments . s Each 
experiment  was  designed  around  a laser-guided  weapon  and 
a corresponding  weapon  delivery  mode.  The  weapons  simula- 
ted in  the  study  were  chosen  from  current  and  potential 
(weapons  under  development)  laser-guided  ordnance  --  GBU-10, 
Laser  Guided  Maverick,  and  Sabre.  The  weapon-delivery 
modes  simulated  were  derived  from  basic  F-16  avionics  ca- 
pabilities. The  weapons  and  weapon  delivery  modes  were 
chosen  to  interact  with  the  FOVs  of  the  simulated  EO  pod  so 
that  each  FOV  pair  could  be  evaluated  under  separate  ac- 
quisition range,  weapons  delivery,  and  target  constraints.  ^ 

For  the  three  experiments,  interdiction  mission  scen- 
arios were  designed  around  a high-threat,  Central  European 
environment  that  included  simulated  visibility  and  weather- 
ceiling restrictions.  Pilots  employed  high-speed  low- 
altitude  egress  to  counter  the  simulated  threat  and  weather 
environment . 

In  each  experiment,  the  three  FOV  pairs  were  evaluated 
at  long,  intermediate,  and  short  aircraft-to-target  ranges. 
The  ranges  were  based  on  typical  visibility  restrictions 
in  the  Central  European  region  and  on  weapons-delivery 
constraints.  The  ranges  were  controlled  through  prebriefed 
pop-up  points  and  the  activation  of  the  EO  pod  video  at 
specific  aircraft-to-target  ranges.  With  three  data  run 
replications  planned  for  each  range  point,  each  experiment 
contained  a total  of  27  data  runs  (3  FOV  pairs  X 3 ranges 
X 3 replications  = 27  data  runs) . 
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Subjective  data  from  each  participating  pilot  were 
obtained  through  questionnaires.  In  the  questionnaires, 
pilots  rated  each  FOV  pair  by  workload  and  system  effect- 
iveness at  the  completion  of  each  experiment  and  at  the 
completion  of  all  experiments.  Objective  data  were  obtained 
from  each  data  run  through  measures  of  specific  parameters 
related  to  pilot  workload  and  system  effectiveness.  Work- 
load measures  were  total  time  to  acquire  (lock  on)  the  tar- 
get and  the  number  of  pod-related  events  to  acquire  (lock 
on)  the  target.  Pod-related  events  consisted  of  pod-sensor 
slewing  events,  target  designation  events,  and  FOV  changes. 
Effectiveness  measures  consisted  of  target  acquisition  and 
weapons  delivery  success  rates.  Other  objective  measures 
were  target  acquisition  and  weapons  delivery  ranges. 

Six  of  eight  pilots  scheduled  to  participate  in  the 
study  completed  all  the  experiments.  Two  pilots  did  not 
complete  the  experiments  because  of  a facility  breakdown. 

The  analysis  of  pilot  ratings  determined  that  the  6°  and 
1.5°  FOV  pair  provided  the  least  pilot  workload  and  the 
most  system  effectiveness.  Analysis  of  the  objective  work- 
load measures  determined  that  the  6°  and  1.5°  FOV  pair  pro- 
vided the  lowest  pilot  workload  over  the  three  experiments. 
Analysis  of  the  objective  effectiveness  measures  did  not 
determine  any  significant  difference  between  FOV  pairs.  This 
was  most  likely  due  to  the  small  subject  population  and  the 
large  variance  between  pilot  performance  levels.  The  trends 
in  the  target  acquisition  and  weapons  release  range  indica- 
ted that  the  6°  and  1.5°  FOV  pair  provided  the  longest  tar- 
get acquisition  and  weapons  release  ranges.  Analysis  of 
narrow-FOV  utilization  determined  that  narrow-FOV  usage  for 
target  acquisition  was  significantly  higher  in  the  1.5° 
narrow  FOV. 

The  pod-sensor  FOR  was  evaluated  through  analysis  of 
pod-seeker  line-of-sight  warning,  obscuration,  and  gimbal 
limit  data.  A simulated  pod-seeker  line-of-sight  obscura- 
tion limit  was  implemented  in  the  simulator  on  the  basis 
of  actual  F-16  aircraft/stores  masking  - with  the  pod  mount- 
ed at  the  lower  right  side  of  the  air  inlet.  For  convenience 
of  analysis,  the  pod-seeker  gimbal  limits  were  defined  from 
150  to  180  degrees  of  azimuth  at  zero  degrees  of  elevation 
(see  Figure  1.1-4,  page  7).  In  addition,  a warning  tone  was 
provided  to  the  pilots  that  had  an  onset  10  degrees  below 
the  obscuration  and  gimbal  limits.  Analysis  of  the  pod- 
seeker  line-of-sight  warning,  obscuration,  and  gimbal  limit 
data  from  the  data  runs  determined  that 


vii 


V 


1.  Pilots  responded  well  to  the  warning  in  that 
out  of  255  warnings  only  90  resulted  in  pod- 
seeker  line-of-sight  obscuration  or  gimbal 
limit . 

2.  The  largest  percentage  of  pod-seeker  gimbal 
limits  occurred  between  160  and  170  degrees 
of  azimuth. 

From  the  results  of  the  FOV  and  FOR  analyses,  it  is 
concluded  that 

1.  Given  the  mission  and  environment  simulated 
and  the  simulator  constraints,  the  6°  and 
1.5°  FOV  pair  provide  the  lowest  pilot  work- 
load and  the  best  system  effectiveness  of  the 
three  FOV  pairs  evaluated. 

2.  Pod-sensor  gimbal  back-look  limits  between  160 
and  170  degrees  accommodated  the  majority  of 
pod-sensor  gimbal  limits  required  for  laser 
guided  bomb  delivery  in  the  study. 

3.  Because  of  excellent  pilot  response  to  the 
pod  line-of-sight  obscuration  and  gimbal  limit 
warning  in  the  study,  consideration  should  be 
given  to  the  implementation  of  a warning  system 
on  aircraft  carrying  an  advanced  EO  pod. 
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SECTION  1 


INTRODUCTION 


The  F-16  advanced  electro-optical  (EO)  pod  fieid-of-view  (FOV) 
study  was  accomplished  by  General  Dynamics  at  Martin-Marietta's 
Simulation  and  Test  Laboratory  (STL)  under  contract  to  the  United 
States  Air  Force,  Aeronautical  Systems  Division  (ASD) . The  study 
was  conducted  to  aid  ASD  in  defining  pod  f ield-of-view  and  field- 
of-regard  requirements  for  a daylight  advanced  EO  pod  employed  from 
an  F-16  type  fighter  aircraft.  The  study  was  confined  to  the  ex- 
amination of  three  FOV  pairs,  provided  by  ASD/AER,  which  are  re- 
presentative of  current  EO  pod  designs.  The  three  FOV  pairs  were 
evaluated  in  a simulated  Central  European  weapons  delivery  environ- 
ment by  TAC  pilots  employing  current  and  potential  laser-guided 
weapons,  weapons  delivery  modes  derived  from  basic  F-16  avionics 
capabilities,  and  up-to-date  tactics.  The  objective  of  the  study 
was  to  determine  which  FOV  pair  created  the  lowest  pilot  workload 
and  the  greatest  effectiveness  in  locating,  acquiring,  and  attack- 
ing targets  in  the  simulated  environment.  In  addition,  pod-sensor 
gimbal  angles  were  investigated  during  target  attacks  from  weapon 
delivery  to  weapon  impact  to  determine  the  pod  f ield-of-regara 
required  by  a pilot  to  successfully  deliver  laser-guided  ordnance. 

Before  the  simulation  tests  were  conducted,  a test  plan 
(Reference  1)  was  prepared  and  submitted  to  the  Air  Force  for 
approval.  Since  this  plan  is  lengthy,  it  is  summarized  in  this 
report  for  convenience  (Section  2)  so  that  along  with  the  infor- 
mation provided  in  the  following  sections  (Sections  3,  4,  and  5), 
a coherent  final  report  would  result.  Also,  details  on  the  stat- 
istical equations  used  to  achieve  the  study  results,  the  supportive 
data  from  which  the  study  results  were  derived,  and  a detailed 
description  of  the  simulation  facility  are  provided  in  three 
appendixes  (Appendixes  A,  B,  and  C,  respectively). 
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SECTION  2 

SIMULATION  DESCRIPTION 


2.1  Simulation  Conf iguration 
2.1.1  Simulation  Facility 

. The  study  was  conducted  at  the  Simulation  and  Test  Laboratory 
(STL)  of  the  Martin  Marietta  Corporation,  Orlando,  Florida.  The 
simulation  was  mechanized  through  an  F-16  cockpit  mockup,  analog  and 
digital  computers,  a terrain  board,  and  an  electro-optical  probe 
system.  The  F-16  cockpit  mockup  configuration  was  based  on  the  full- 
scale  development  (FSD)  aircraft  and  contained  the  controls  and  dis- 
plays required  to  operate  the  aircraft  and  deliver  various  laser- 
guided  weapons  employing  an  electro-optical  pod.  The  major  controls 
and  displays  employed  in  the  cockpit  mockup  are  depicted  in  Figure  2.1-1. 

The  computer  software  implemented  in  the  study  consisted  of  an 
F-16  aerodynamic  model  adjusted  for  representative  ordnance  load;  a 
model  of  the  fly-by-wire  flight  control  computer;  a model  of  the 
integrated  digital  avionics  system;  and  a model  of  the  advanced  EO 
pod,  which  included  a SLAVE  mode  (pod  sensor  line-of-sight  slaved  to 
preselected  coordinates  or  to  the  aircraft's  velocity  vector),  an  area 
correlation  track  mode,  and  a contrast  track  mode.  Also  implemented  in 
the  computers  was  the  fire  control  logic,  release  constraints,  ballis- 
tics, and  heads-up  display  (HUD)  symbology  for  each  weapon  system  and 
delivery  mode  simulated. 

The  terrain  board  was  80-feet  long  by  40-feet  wide  which,  at  a 
scale  of  1200  to  1,  provided  a 16  n.mi.  by  8 n.mi.  area  of  simulated 
terrain.  Mirrors  surround  the  terrain  to  provide  the  illusion  of 
terrain  extension.  Motion  excursion  limits  implemented  in  the  compu- 
ters to  prevent  the  electro-optical  probe  system  from  damaging  the 
mirrors  limit  the  working  area  on  the  terrain  board  to  an  approximate 
14  n.mi.  by  7 n.mi  area.  The  topography  of  the  terrain  board  was 
characteristically  Central  European  and  contained  all  the  natural  and 
man-made  features  normally  found  in  that  region.  A two-probe  EO  system 
was  suspended  over  the  terrain  board  by  a gantry  that  was  driven  in 
combination  with  the  terrain  board  to  provide  the  flight  simulation. 

The  orientation  of  the  two  probes  was  fixed  longitudinally.  The  North 
probe  provided  the  video  for  the  windscreen  display  at  the  cockpit  mock- 
up.  The  South  probe,  which  was  located  1 foot  (1200  scale  feet)  behind 
and  2 inches  (200  scale  feet)  below  the  North  probe,  provided  the 
simulated  EO  pod  video  at  the  cockpit  mockup. 

An  experimenter's  console  was  also  provided  at  the  cockpit 
mockup.  The  console  contained  the  necessary  computer  controls, 
video  displays,  and  communication  lines  for  the  experimenter  to 
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monitor  and  conduct  all  familiarisation,  training,  and  exoerimenta- 
tion.  Photographs  of  the  simulation  facility  are  provided  in 
Figures  2.1-2  and  2.1-3.  A detailed  discussion  of  the  simulation 
facility  is  contained  in  Appendix  C. 


2.1.2  Field-of-View  Pairs 


the 


The  following 
study : 

1.  6 degrees 

2.  6 degrees 

3.  3 degrees 


three  f ield-of- view  pairs  were 


and  1.5  degrees 
and  0.86  degree 
and  0.86  degree 


implemented  in 


The  original  intent  of  the  study  was  to  evaluate  a 1.5-degree 
and  a 1-aegree  narrow  FOV.  Underscanning  the  1.5-degree  FOV  to 
achieve  a 1-degree  FOV  was  attempted;  however,  the  resultant 
video  was  unusable.  Therefore,  a new  lens  which  provided  an  0.86- 
degree  FOV  was  manufactured  by  Martin  Marietta.  Although  the 
video  clarity  provided  by  the  0.86-degree  lens  was  degraded  as 
compared  with  the  1.5-degree  FOV,  it  was  usable  and  was  as  close 
to  the  desired  1-degree  FOV  as  was  possible  given  the  time  and 
cost  constraints  of  the  study. 


2.1.3  Fie Id- of -Regard 

For  the  purposes  of  the  simulation,  the  assumed  mounting 
location  for  the  pod  was  on  the  low7er  right  side  of  the  F-16's 
air  inlet  (pod-seeker  gimbal  located  at  FS-178.24,  \vL-4S,  BL-20) . 

At  this  location,  the  pod  sensor  masking  limits  were  as  described 
in  Figure  2.1-4,  assuming  gimbal  limits  between  150  and  180  degrees 
of  azimuth  at  zero  degrees  of  elevation.  Equations  which  approxi- 
mated the  true  masking  limits  were  developed  and  implemented  in  the 
computer,  which  veilded  the  simulated  aircraft/stores  limits  de- 
picted in  Figure  2.1-4.  In  addition,  an  aircraft/stores  masking 
and  gimbal  limit  warning  which  provided  a warning  tone  to  the 
pilot  10  degrees  below  the  simulated  limits  was  implemented  in  the 
simulator.  The  warning  onset  is  also  depicted  in  Figure  2.1-4. 


2.1.4  Weapons  and  Weapons  Delivery  Modes 


Weapon 
laser-guided 
teristics;  a 


simulations  implemented  for  the  study  cor.sis 
bomb  (LGB)  with  MK-8A/G3U-10  ballistics  and 
standoff-  ranee  laser-guided  missile  (LGM) 
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guided  Maverick  characteristics;  and  a short-range,  light-weight, 
laser-guided  missile  (LWLGM)  with  Sabre  characteristics. 


Weapons  delivery  modes  for  the  study  were  created  from  basic 
F-16  avionics  capabilities  to  support  the  weapon  system  and  mis- 
sions chosen  for  the  study.  The  standard  F-16  Continuously  Com- 
puted Release  Point  (CCRP)  weapons  delivery  mode  was  simulated 
for  the  delivery  of  laser-guided  bombs  against  prebriefed  tar- 
gets. This  mode  provided  head-up  and  head-down  cueing  and  stee- 
ring to  a preprogrammed  set  of  target  coordinates  and  head-up 
weapons  release  solution  cues  for  weapons  delivery.  Employing 
this  mode,  the  pilot  had  the  capability  to  toss,  to  deliver 
level,  or  to  dive-deliver  the  weapon  at  the  target. 

A candidate  laser-guided  Maverick  head-up  delivery  mode  was 
simulated  for  attacking  prebriefed  targets.  Missile  acquisition 
and  launch  symbology  was  implemented  on  the  head-up  display  for 
weapon  delivery,  and  target  cueing  was  provided  on  the  head-up 
and  head-down  displays  as  in  the  CCRP  weapons  delivery  mode. 

A candidate  visual  target-acquisition  weapon  delivery  mode 
for  attacking  targets  of  opportunity  was  simulated  for  the  de- 
livery of  the  light-weight  laser-guided  missile.  In  this  mode 
the  seeker  head  of  the  missile  and  line  of  sight  of  the  EO  pod 
were  initially  slaved  to  the  flight  path  of  the  aircraft.  Because 
this  technique  is  found  in  the  dive-toss  weapons  delivery  modes 
on  aircraft,  this  mode  was  named  the  Laser  Guided  Missile  Dive 
Toss  (LGMDT)  weapons  delivery  mode.  Missile  acquisition  and 
launch  symbology  was  implemented  on  the  head-up  display  as  in  the 
Maverick  mode. 

In  addition  to  the  weapons  delivery  modes,  the  F-16  naviga- 
tion (NAV)  mode  was  simulated  to  provide  the  pilot  with  the  means 
to  steer  to  a waypoint  prior  to  target  attack.  The  weapons,  wea- 
pons delivery  modes,  and  navigation  mode  simulated  for  this  study 
are  summarized  in  Figure  2.1-5. 

2.1.5  EO  Pod/Weapon/Weapon  Delivery  Mode  Interaction 

The  weapons,  weapon  delivery  modes,  and  attack  scenario  em- 
ployed in  the  study  were  chosen  to  have  a specific  impact  on  the 
requirements  for  an  EO  pod.  The  LGB/CCRP  combination  was  chosen 
to  investigate  the  impact  of  aided  target  acquisition  (target 
cueing)  and  weapons  delivery  from  medium  to  short  ranges  on  the 
pod  sensor  wide  and  narrow  fields  of  view.  Since  the  LGB  has  the 
longest  time  of  flight,  it  was  also  chosen  for  its  impact  (follo- 
wing weapon  release)  on  the  pod  sensor  f ield-of-regard  require- 
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merits.  The  LG  Maverick/LGM  combination  was  chosen  to  investigate 
the  impact  of  aided  target  acquisition  and  weapon  delivery  from 
stand-off  ranges  on  the  EO  pod  sensor  narrow  fields  of  view.  The 
LUTLGM/LGMDT  combination  was  chosen  to  investigate  the  impact  of 
unaided  (visual)  target  acquisition  and  weapons  delivery  from 
close  ranges  on  the  EO  pod  sensor  wide  fields  of  view.  The  EO 
pod/weapon/weapon  delivery  mode  interactions  are  summarized  in 
Table  2.1-1. 


2.2  Experimental  Method 

The  experimental  design,  the  evaluation  basis,  and  the  data 
retrieval  system  employed  in  the  study  are  described  briefly  below. 
Details  are  given  in  Reference  1,  Sections  4 through  7,  pages  20 
through  58. 

2.2.1  Scenarios  for  FOV  Evaluation 

Interdiction  missions  were  simulated  in  the  study.  For 
laser  guided  bombs,  a deep  interdiction  mission  (40  miles  behind 
the  FEBA  (Forward  Edge  of  the  Battle  Area)  against  prebriefed 
fixed  targets  was  simulated.  Targets  consisted  of  bridges,  POL 
storage  tanks,  aircraft  hangers,  etc.  For  the  laser-guided 
Maverick,  two  interdiction  missions  were  simulated  --  interdiction 
(1-5  miles  behind  the  FEBA)  against  groups  of  armored  vehicles  in 
battle  arrays,  and  interdiction  (40  miles  behind  the  FEBA)  against 
both  armored  vehicles  in  battlefield  and  linear  arrays  and  fixed 
hardened  targets  (aircraft  revetments,  bunkers,  etc).  For  the 
LWLGM,  an  armed  reconnaissance  mission  behind  the  FEBA  against 
armor  in  both  linear  and  battlefield  arrays  was  simulated. 

With  the  exception  of  the  Maverick  shallow  interdiction  mis- 
sion, the  mission  profile  was  low-altitude  (500  feet  AGL) , high- 
speed (540  KTAS)  ingress,  pop-up  for  target  acquisition  and  weapons 
delivery,  and  low-altitude  (500  feet  AGL)  egress.  The  ingress 
altitude  was  set  at  500  feet  in  lieu  of  multiple  altitudes  of  250, 
500,  and  1000  feet  owing  to  the  need  to  limit  the  length  of  the 
tests  because  of  simulator  facility  availability.  The  Maverick 
shallow  interdiction  mission  was  conducted  at  2000  feet  and  540 
KTAS  for  ingress  so  that  target  acquisition  could  be  examined  at 
long  range.  The  attack  scenarios  employed  for  each  weapon  and 
weapon  delivery  mode  are  depicted  in  Figures  2.2-1  through  2.2-4. 

A Central  European  environment  was  simulated.  The  simulated 
environment  included  LTest  German  topography  on  the  model  board, 
limited  visibility  (3  to  6 n.mi) , and  a low  weather  ceiling 
(2500  feet  MSL) . Since  actual  visibility  restrictions  (smoke, 
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haze,  dust,  etc.)  could  not  be  simulated,  target  visibility  was 
denied  to  the  pilot  by  using  terrain  masking  during  the  low-altitu 
ingress  and  by  turning  the  video  off  to  the  radar/electro-optical 
(REO)  display  until  a specific  range  to  target  was  reached.  The 
weather  ceiling  was  simulated  by  briefing  the  pilots  to  stay  below 
1530  feet  MSL  in  their  pop-up  maneuvers  and  by  penalizing  them 
by  scoring  the  run  unsuccessful  when  they  flew  above  the  briefed 
altitude . 


- . - . - w.-.T;  c ^ - — c 


design 


The  study  consisted  of  three  experiments.  Each  experiment  was 
designed  around  a weapon  and  the  corresponding  weapon  delivery 
node"--  LG3/CCRP,  LG  Maverick/LGM,  and  LlvLGM/ LGMDT- - employing'  the 
attack  scenarios  described  above.  In  each  expierment,  the  pilot's 
ability  to  employ  each  of  the  FOV  pairs  at  three  pop-up/REO  dis- 
play activation  ranges  was  investigated.  The  pop-up  and  REO  dis- 
play activation  ranges  were  based  on  weapon  capabilities  and  typi- 
cal weather  phenomena  for  the  Central  European  environment.  The 
detailed  rationale  for  pop-up  and  REO  display  activation  ranges  in 
each  experiment  is  given  in  Table  2.2-1.  Three  replications  were 
planned  for  each  FOV  pair  and  pop-up/REO  activation  range  combina- 
tion. The  resultant  experimental  matrix  for  each  experiment  had 
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data  runs 

(3  FOV  pairs  X 

3 pop-up/REO 

ac 

tivation  ranges  X 3 

ep 

lications 

= 27 

data  runs) 

. The  total 

of 

three  experiments  thus 

on 

tained  81 

data 

runs  (3  experiments  X 

27 

data  runs/ experiment) . 

So  that 

the 

results  of 

the  simulati 

on 

would  not  be  influenced 

by  unrealistic  inertial  target  accuracy  or  by  pilot  memorization 
of  target  locations,  additional  parameters  were  varied.  An  error 
of  1000  feet  was  introduced  with  random  azimuth  for  each  pre-cued 
target  to  simulate  inertial  error  and  coordinate  uncertainty.  This 
error  assumed  an  inertial  update  within  5 minutes  of  the  target 
with  an  inertial  drift  rate  of  1 n. mi/hr.  Twelve  targets  were 
assigned  to  each  of  the  three  experiments  (36  separate  targets). 

To  keep  target  study  to  an  acceptable  level,  four  separate  targets 
were  assigned  to  the  nine  data  runs  for  each  FOV  pair.  During  the 
nine  data  runs  for  each  FOV  pair  in  an  experiment,  the  four  tar- 
gets were  ordered  such  that  that  pilot  would  not  attack  the  same 
target  w’ithout  first  attacking  two  other  targets.  No  target  was 
attacked  more  than  three  times.  When  the  same  target  was  attacked 
mere  than  once,  the  run-in  heading  was  varied  at  least  30  degrees 
so  that  the  pilot  never  achieved  the  identical  target  aspect. 

In  each  of  the  three  experiments,  nine  sets  of  conditions 
were  tested,  each  representing  a unique  combination  of  FOV  pair 
and  pop-up/REO  activation  range.  Each  set  of  conditions  was  re- 
peated three  times  against  different  targets.  The  effects  of  tar- 
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gee  order  and  difficulty  on  the  data  were  removed  by  systematically 
counterbalancing  the  nine  conditions  over  the  run  matrices  for 
three  sets  of  two  pilots.  Therefore,  six  pilots  had  to  complete 
all  81  of  the  data  runs  to  fully  counterbalance  the  data.  The 
counterblanaced  run  matrices  for  each  experiment  are  shown  in 
Tables  2.2-2  through  2.2-4. 

2.2.3  Subject  Pilots 

A total  of  eight  subject  pilots  were  requested  from  the  United 
States  Air  Force  Tactical  Air  Command  to  participate  in  the  study. 
The  request  covered  four  A-7  qualified  pilots  from  operational 
squadrons  and  four  F-4  qualified  pilots,  two  from  an  operational 
squadron  at  Eglin  AFB,  Florida,  and  two  from  Nellis  AFB , Nevada. 
Pilots  were  to  be  randomly  selected  with  the  exception  that  none 
were  to  have  experience  flying  the  F-16  aircraft.  The  mix  of  A-7 
and  F-4  pilots  was  requested  to  obtain  pilots  with  and  without 
head-up-display  (HUD)  experience.  The  eight  subject  pilots  are 
listed  in  Table  2.2-5  along  with  their  experience. 

2.2.4  Evaluation  Basis 


The  evaluation  of  the  FOV  pairs  was  based  on  two  data  sources 
--  subjective  data  elicited  from  the  pilots  through  questionnaries 
and  objective  data  taken  from  parameter  readouts  provided  by  the 
simulation  computer  after  each  of  the  data  runs.  Because  of  the 
brevity  of  the  experiments  (small  subject  population  and  small 
number  of  replications  for  each  data  point) , the  primary  data 
source  was  the  subjective  data. 

For  the  FOV  pair  evaluation,  both  data  sources  were  broken 
into  two  categories  - pilot  workload  and  system  effectiveness. 

For  the  subjective  analysis,  pilots  rated  workload  and  system 
effectiveness  for  each  FOV  pair  in  each  weapon  delivery  mode. 

The  collective  ratings  were  analyzed  to  determining  which  FOV 
pair  provided  the  least  workload  and  the  most  effectiveness 
over  the  three  weapon  delivery  modes.  For  the  objective  analysis, 
separate  measures  were  employed  to  determine  pilot  workload  and 
system  effectiveness.  Pilot  workload  was  evaluated  by  two  mea- 
sures. The  first  was  to  determine  the  time  required  by  the  pilot 
to  locate,  acquire  (lock-on)  and  identify  the  target  employing 
each  FOV  pair.  Included  in  this  measure  was  total  time  to  locate 
acquire  and  identify  the  target  in  both  wide  and  narrow  FOVs  and 
the  time  spent  in  each  wide  and  narrow  FOV.  Since  the  pilot  had 
to  physically  operate  discrete  switches  and  controls  to  search 
for,  acquire  and  verify  the  target,  the  second  workload -measure 
was  to  sum  the  number  of  discrete  switch  and  control  operations 
related  to  the  pod  during  this  phase  of  the  mission.  These  opera- 
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Table  2.2-2  COUNTERBALANCED  LGB/CCRP  RUN  MATRICES 


Note:  Number  inside  matrices  indicate  sequence 
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Table  2.2-3  COUNTERBALANCED  LG  MAVERICK/LGM  MATRICES 

Note:  Numbers  inside  matrices  indicate  sequence 

in  which  data  runs  were  accomplished. 
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Table  2.2-4 
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Note:  Numbers  inside  matrices  indicate  sequence 

in  which  data  runs  were  accomplished. 
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tions  included  pod  sensor  sieving,  target  designation  events,  and 
pod  FOV  changes. 

System  effectiveness  with  each  FOV  pair  was  evaluated  by  two 
measures  --  target  acquisition  success  rates  and  weapons  delivery 
success  rates.  Target  acquisition  success  rates  were  used  to 
measure  system  effectiveness  during  initial  target  search,  acqui- 
sition and  verification.  Weapons  delivery  success  rates  were  used 
to  measure  system  effectiveness  after  initial  target  acquisition 
during  weapons  delivery  and  the  designation  turn.  Target  acquisi- 
tion success  was  based  on  whether  or  not  the  pilot  found  the 
correct  target  and  locked  the  pod  on  it  outside  the  minimum  wea- 
pons delivery  range.  The  determination  of  weapons  delivery  suc- 
cess or  failure  was  much  more  complex.  The  conditions  for 
success  were  based  on: 


1.  Meeting  weapon  release  constraints 

2.  Tracking  the  target  consistently  (laser  aiming)  during 
the  weapon's  time-of-f light 

3.  Keeping  the  laser  "Oh”’  consistently  during  the  weapon's 
time-of-f light. 


Pod  sensor  f ield-of-regard  requirements  were  evaluated 
through  an  analysis  of  pod  line-of-sight  warning  and  obscuration 
data.  These  data  were  applied  to  the  aircraft  masking  and  pod 
gimbal  limits  employed  in  the  study  to  define  the  field  of  re- 
gard required  by  pilots  to  deliver  laser-guided  weapons  from  the 
F-16  aircraft. 

2.3  Data  Categories  Evaluated 

A brief  description  of  the  subjective  and  objective  data 
evaluated  in  the  study  is  given  below.  A complete  description  of 
the  data  collected  during  the  course  of  the  experiments  is  given 
in  Reference  1,  Section  7 and  Appendix  B. 


2.3.1  Subjective  Data 


Three  subjective  data  categories  were  evaluated  in  the  study: 
pilot  comments  and  observations,  pilot  FOV  pair  workload  ratings, 
and  pilot  FOV  pair  effectiveness  ratings.  All  subjective  data  were 
collected  on  debriefing  forms  employed  after  each  data  run,  after 
each  simulation  period,  after  the  completion  of  each  experiment, 
and  after  the  completion  of  all  experiments. 
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Pilot  comments  and  observations  were  summarized  both  from 
their  direct  responses  to"questions  in  the  debriefing  materials 
and  from  their  unsolicited  remarks  regarding  the  pod  FOV  pairs. 

I Following  the  completion  of  each  experiment  and  after  all  experi- 

ments were  completed,  the  pilots  rated  the  relative  workload  and 
effectiveness  associated  with  each  FOV  pair  on  a scale  ranging 
from  0.0  to  10.0  for  a series  of  four  tasks: 

# 

1.  Search  for  targets 

2.  Verify  targets 

3.  Fly  designation  turn  and  maintain  tracker  on  target 

4.  Perform  complete  mission  (summary  of  all  tasks). 

Included  in  the  ratings  was  one  non-pod-related  task,  weapon  de- 
livery. An  example  of  the  questionnaire,  showing  the  rating  scale, 
is  presented  in  Figure  2.3-1.  Data  were  retrieved  from  the  scales, 
which  were  exactly  10  cm  long,  by  measuring  the  distance  from  the 
left-hand  index  to  the  point  at  which  the  pilot  had  indicated  his 
rating  for  each  FOV  pair. 

2.3.2  Objective  Data 

The  objective  data  were  subdivided  into  the  following  four 
categories : 

1.  Workload  data 

2.  Effectiveness  data 

3.  Other  data  (data  collected  to  evaluate  the  FOV  pairs 
which  did  not  fit  either  workload  or  effectiveness 
categories) 

4.  Field-of-regard  data. 

The  majority  of  the  objective  data  was  taken  from  parameters  mea- 
sured by  the  computer  during  each  data  run  and  printed  immediately 
after  the  run.  A sample  end-of-run  computer  printout  with  pertinent 
events  highlighted  is  shown  in  Figure  2.3-2. 

2. 3.2.1  Workload  Data 

As  previously  discussed,  the  workload  data  consisted  of  tar- 
get acquisition  time  lines  and  pod-related  events.  In  both  the 
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Figure  2.3-1  Debriefing  Questionnaire  Example 
(not  to  scale) 


On  those  questions  requiring  a subjective  rating,  place  an  X and 
the  field  of  view  pair  (6°/l^°,  6°/l°,  3°/l°)  at  the  appropriate 
8 pot  along  the  rating  line. 


ExasiDle : 


Least 
Effective 


100% 

Effective 


FOV  Effectiveness 

1.  Rate  the  performance  effectiveness  of  each  FOV  pair  for 
each  of  the  following  tasks. 


Effectiveness 


Perform  Complete  Mission 
Check  Waypoints  (LGMDT) 
Search  for  Targets 
Verify  Targets 
Wpn  Delivery 
Fly  Designation  Turn 


Other 


Least 

.Effective 


100% 

Effective. 
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Figure  2.3-2  find- of- Run  Dn  t <1  Fisting  fixnmple 


LGB/CCRP  and  LG  Maverick/LGM  experiments,  the  tine  to  search, 
acquire  (lock-on)  and  verify  the  target  was  determined  in  each  run 
by  measuring  the  time  from  REO  activation  to  target  acquisition. 
Acquisition  of  the  target  was  determined  by  the-  experimenter,  who 
placed  a "MARK"  in  the  data  when  the  pilot  designated  (locked  the 
pod  tracked  on)  the  correct  target.  The  pod-related  events  per- 
formed to  search,  acquire  (lock-on),  and  verify  the  target  were 
summed  from  REO  activation  to  target  acquisition. 

In  the  LlvLGM/LGMDT  experiment,  the  REO  display  was  on  through- 
out the  run;  therefore,  a different  technique  for  measuring  acquisi- 
tion time  and  pod-related  events  was  applied.  Following  the  pop- 
up maneuver,  pilots  did  not  go  head-down  and  begin  slewing  the  pod 
sensor  to  search  for  the  target  until  they  located  the  suspected 
target  area  heads-up  on  the  windscreen  display.  Therefore,  the 
time  to  search,  verify,  and  acquire  the  target  employing  the  EO 
pod  was  measured  from  the  first  slewing  event  following  the  pop-up 
maneuver  to  target  acquisition.  Pod-related  events  were  measured 
by  the  same  criteria. 

2. 3. 2. 2 Effectiveness  Data 

Effectiveness  data  consisted  of  acquisition  and  weapons  de- 
livery success/ failure  data  for  each  run  accomplished  over  the 
three  experiments.  Acquisition  success  was  initially  determined 
at  the  experimenter's  console,  where  he  placed  a "MARK"  in  the 
data  when  the  pilot  successfully  acquired  (locked-on)  the  correct 
target  with  the  EO  pod.  The  acquisition  success  was  later  con- 
firmed by  reviewing  the  data  printout  to  determine  if  the  acqui- 
sition was  made  outside  minimum  weapons  delivery  constraints. 

In  a few  instances,  either  the  experimenter  forgot  to  insert  the 
mark  in  the  run  event  data  or  the  computer  malfunctioned  and  did 
not  produce  a good  end-of-run  printout.  In  these  instances,  the 
video  tape  made  during  the  run  was  reviewed  to  ascertain  the 
acquisition  success / failure . 

Weapons  delivery  success  was  determined  after  examination 
of  the  data  from  the  individual  runs.  During  a run,  the  experi- 
menter would  watch  for  specific  events  and  conditions  to  occur. 

These  included  successful  target  acquisition,  weapons  release, 
consistency  of  target  tracking,  consistency  of  laser  designation, 
and  obscuration  warning  cues.  When  a run  appeared  to  be  success- 
ful on  the  basis  of  observation,  a mark  was  entered  in  the  run 
event  data  at  the  conclusion  of  the  run  and  logged  in  the  experi- 
menter's log  as  successful.  Also  logged  were  any  events  or  condi- 
tions which  might  make  the  run  unsuccessful  (pod  obscuration 
warnings,  late  pickle,  loss  of  target  lock-on,  target  overflight, 
etc).  For  all  runs  logged  as  successful,  data  printouts  were  ex- 
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amined  to  determine  their  validity.  Conditions  for  a successful 
weapons  release  are  described  below. 

(1)  Weapon  Release  Conditions 

(a)  LGB  - The  following  formulas  were  employed  to 
calculate  the  maximum  steering  error  and  maximum 
weapon  release  delay  that  could  be  tolerated  by  the 
weapon  seeker  head.  The  formulas  were  derived 
from  information  provided  by  Texas  Instruments, 

Inc.,  manufacturer  of  the  GBU-10  guidance  kits. 

Steering  Error  H Arc  Sin  ^ 600/  [ Rb  “ (8.0  R^/Tf)  Cos  GJ 
where 

600  = 600  ft  allowable  azimuth  error  with  an 
8-sec  weapon  guidance  time 
R^  = Release  range  (ft) 

Tf  = Weapons  time-of-f all  at  weapon  release  (sec) 

9 = Aircraft  pitch  angle  at  weapon  release 

Weapon  Release  < Computed  Weapons  Release  + 0.7  sec 

(b)  LG  Maverick  and  LWLGM  - Maximum  seeker  of f-boresight 
launch  limits  and  maximum/minimum  launch  ranges  for 
the  proposed  LG  Maverick  and  Sabre  missile  systems 
were  implemented  in  the  analysis.  Classification 

of  this  information  prevents  its  publication  in  this 
document . 

(2)  Target  Tracking  Requirements  after  Weapons  Release 

(a)  Tracker  drift  tolerance  when  pod  tracker  broke  lock 
during  laser  designation  - 100  feet. 

(b)  Time  tolerance  when  pod  tracker  broke  lock  or  when 
pod  line-of-sight  became  obscured  during  laser 
designation  - 1 second. 

(c)  Target  laser-designation  requirements  during  weapon 
terminal  guidance  --  stabilized  for  final  2 seconds 
of  weapon  time  of  flight  (target  dependent) . 

(3)  Laser  Fire  Requirements  - Laser  fire  not  interrupted 

more  than  1 second. 
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When  runs  were  questionable  in  this  analysis,  the  final  deter- 
mination for  success/failure  was  made  in  a review  of  the  video 
tapes . 

2. 3. 2. 3 Other  Data 

Two  other  measures  were  applied  to  the  FOV  pair  evaluation. 
Target  acquisition  and  weapon  release  ranges  were  evaluated  to 
determine  if  one  of  the  FOV  pairs  provided  a range  advantage. 

The  acquisition  and  weapon  release  range  data  were  taken  directly 
from  the  end-of-run  data  printouts.  A narrow-FOV  employment 
analysis  was  conducted  to  determine  how  frequently  the  narrow  FOV 
was  employed  in  each  FOV  pair,  and  for  what  purpose.  These  data 
were  derived  from  a review  of  all  runs  in  which  a successful 
target  acquisition  was  achieved  and  the  narrow  FOV  was  employed 
during  the  target-acquisition  phase. 

2. 3. 2.4  Field-of-Regard  Data 

Pod  sensor  gimbal  angles  were  measured  at  the  point  of  sen- 
sor line-of -sight  warning  onset  and  at  the  point  of  aircraft/stores 
masking  or  gimbal  limit  during  each  run.  Both  data  sets  were 
retrieved  from  the  end-of-run  data  printout. 
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SECTION  3 


DATA  ANALYSIS  METHODS 


3.1  Subjective  Data  Analysis  Methods 

The  significant  effects  in  the  pilot  FOV  pair  ratings  created 
by  the  independent  variables  of  the  study  were  determined  by  both 
parametric  F-tests  (Reference  2)  and  non-parametric  Friedman  Tests 
(Reference  3).  Ordinarily,  only  non-parametric  methods  of  analysis 
would  be  used  for  the  analysis  of  subjective  data.  However,  for 
the  complex  experiments  of  this  study,  the  assumptions  implicit  in 
the  non-parametric  analysis  methods  are  often  violated.  W.  J. 
Conover,  a non-parametric  statistical  advocate,  concludes  a discus- 
sion of  the  Friedman  test  by  saying  that  for  complex  experimental 
situations  one  is  practically  forced  to  use  parametric  tests 
(Reference  4).  The  most  appropriate  parametric  tests  for  the  data 
after  it  was  determined  to  be  ordinal  and  normal  in  distribution 
were  the  F-tests.  Since  some  statisticians  may  argue  that  the  data 
is  non-parametric  from  the  standpoint  that  it  is  not  derived  from 
interval  scale  measurements,  the  non-parametric  Friedman  test  was 
also  applied  to  the  data.  Details  of  both  tests  are  contained  in 
Appendix  A. 

The  two  tests  were  applied  to  both  workload  and  effectiveness 
data,  with  the  following  independent  variables  being  common  to  both: 

1.  Subject  pilots  (6). 

2.  Fie ld-of- view  pairs  (3). 

3.  Pod-related  tasks  --  workload  (3),  effectiveness  (4). 

4.  Rating  parameters  (4)  --  the  rating  parameters  consisted 
of  the  FOV  pair  ratings  for  each  experiment  and  "overall" 
FOV  pair  ratings  completed  by  each  pilot  during  the  end- 
of-study  debriefing. 

For  significance,  the  null-case  hypothesis  (probability  that  the 
effects  due  to  the  independent  variables  were  random)  was  tested 
for  a probability  of  less  than  or  equal  to  0.05  for  both  tests. 

When  the  Friedman  test  determined  significance  due  to  an  inde- 
pendent variable,  the  source  of  the  significance  was  determined  by 
employing  the  Friedman  test  between  each  rating  condition  within  the 
independent  variables.  When  the  F-test  determined  significance  due 
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to  an  independent  variable,  a T-test  (Reference  2)  was  used  to 
determine  the  source  of  the  significance.  The  details  of  the  T- 
test  are  contained  in  Appendix  A. 

Employing  the  F-test,  significant  interactions  between  the  in- 
dependent variables  could  be  determined.  For  significance,  the 
null-case  hypothesis  (probability  that  the  effects  due  to  the  inter- 
actions between  the  independent  variables  were  random)  was  tested 
for  a probability  of  less  than  or  equal  to  0.05.  When  significance 
was  detected,  a graphical  analysis  was  employed  to  determine  the 
source  of  the  interaction  and  its  effect  on  the  main  effect(s). 

An  analysis  of  the  pilot's  comments  and  observations  was  con- 
ducted to  determine  a consensus  of  opinions  about  each  individual 
field  of  view  employed  in  the  study  and  the  field  of  regard  required 
to  deliver  laser-guided  ordnance. 

3.2  Objective  Data  Analysis  Methods 

The  objective  data  analysis  required  two  separate  analysis 
methods  because  of  the  nature  of  the  data.  The  success/ failure 
data  for  target  acquisition  and  weapons  delivery  were  discrete, 
whereas  the  time,  event,  and  range-measurement  data  were  continuous. 
Therefore,  individual  analysis  techniques  were  applied  to  the 
discrete  and  continuous  data. 

3.2.1  Discrete  Data 


The  discrete  target  acquisition  and  weapons  delivery  success/ 
failure  data  had  a very  limited  range  due  to  the  small  number  of 
replications  performed  for  each  data  point.  In  addition,  the  small 
number  of  pilots  participating  in  the  study  created  a large  variance 
in  the  success / failure  data  across  the  experiments.  Thus,  because 
of  the  small  range  in  the  data  and  the  variance  of  pilot  perform- 
ance, the  assumption  of  normality  implicit  in  most  statistical 
analysis  techniques  could  not  be  made,  confining  the  analysis  to 
techniques  which  do  not  assume  normality.  The  data  were  analyzed 
by  use  of  an  analysis  of  variance  for  ranked  data  --  the  Friedman 
Test.  With  this  test,  the  analysis  wTas  limited  to  determining 
significant  effects  in  the  data  due  to  the  following  independent 
variables  in  each  experiment: 

1.  Subject  (6) 

2.  Field-of-View  Pair  (3) 

3.  Pop-up/REO  activation  Range  (3). 
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For-  significance,  the  null-case  hypothesis  (probability  that  the 
efrects  in  the  success/failure  data  due  to  the  independent  variables 
were  random)  was  tested  for  a probability  less  than  or  ecual  to 
0.05. 


3.2.2  Continuous  Pats 

Because  of  the  many  empty  cells  in  the  continuous  data  created 
each  time  a pilot  failed  to  acquire  the  target,  normal  statistical 
analysis  techniques  could  not  be  applied  to  these  data.  However,  a 
trend  analysis  was  accomplished  on  the  mean  performance  level  across 
the  pilots  for  each  pop-up/REO  activation  range  in  each  experiment 
to  determine  if  these  data  supported  the  subjective  results.  The 
analysis  w a s based  on  the  difference  between  the  mean  performance 
levels  and  the  deviation  within  the  mean  performance  measures. 


32 


SECTION  4 


r 


STUDY  RESULTS 


4.1  Summary  of  Experiments 

Six  pilots  completed  162  valid  data  runs  in  each  experiment. 

Two  pilots  did  not  complete  the  study  because  of  a facility  failure. 
Their  data,  which  consisted  of  30  data  runs  accomplished  in  the 
LG3/CCRP  experiment,  were  not  used  to  derive  the  study  results. 

This  did  not  invalidate  the  experimental  results  since  the  study 
required  only  six  pilots  to  complete  the  experiments  and  fully 
counterbalance  the  data  over  the  experimental  variables. 

4.2  Subjective  Data  Results 

4.2.1  Pilot  Comments  and  Observations 

A consensus  of  pilot  comments  and  observations  regarding  each 
field  of  view  employed  in  the  study  and  the  field  of  regard  required 
by  pilots  to  employ  laser-guided  weapons  is  provided  in  Table  4.2-1. 

4.2.2  Pilot  Workload  Assessment 

The  F-test  and  the  Friedman  test  determined  one  effect  due  to 

the  independent  variables.  Of  the  four  independent  variables 

(subjects,  FOV  pair,  task,  and  rating  parameter),  FOV  pair  was  the 

single  significant  influencing  factor  on  pilot  workload  as  rated 

by  the  pilots.  The  mean  of  the  pilot  workload  ratings  for  each  FOV 

?airQis  depicted  in  Figure  4.2-1.  As  shown,  the  6 and  1.5  , 6 and 

0.S6  , and  3 and  0.86  FOV  pairs  were  rated  by  the  pilots  in  an 

ascending  order  of  pilot  workload,  respectively.  The  3°  and  0.86° 

FOV  oair  was  obviously  the  highest  in  workload.  The  difference 
* O " o ° o o 

between  the  6 and  1.5  and  the  6 and  0.86  FOV  pairs  was  not  as 

obvious;  however,  the  T- tests  and  the  Friedman  tests  determined  a 

significant  difference  between  the  pilot  ratings  for  each  FOV  pair. 

The  detailed  results  of  the  F-test,  T-tests,  and  Friedman  tests 

for  the  workload  analysis  are  contained  in  Appendix  B. 

The  F-test  determined  one  effect  due  to  the  interaction  between 
independent  variables.  This  effect  was  due  to  the  interaction  of 
FOV  pair  by  task.  The  graphical  analysis  of  this  interaction  is 
depicted  in  Figure  4.2-2.  In  the  6 and  1.5  and  the  6 and  0.86 
FOV  pairs,  the  pilots  rated  the  verify- targets , fly-designation- 
tura,  perform-complete-mission , and  search- for- targets  task  in  a 
descending  order  of  relative  workload.  In  the  3 and  0.86°  FOV  pair, 
this  trend  reversed,  as  shown  in  Figure  4.2-2  by  the  crossover  between 
data  points.  However,  the  basic  trend  of  rating  the  6°  and  1.5  , 
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SUBJECTIVE  FOV  WORKLOAD  RATINGS 
(Primary  Effect) 


10j(Highest  Workload) 


Figure  4.2-1  Pilot  Workload  Ratings  for  Each  FOV  Pair 


the  6°  and  0.86°,  and  the  3°  and  0.86°  FOV  pairs  in  an  ascending 
order  of  relative  workload  for  each  task  did  not  change . There- 
fore, the  interaction  between  FOV  pair  and  task  variables  had  no 
confounding  effect  on  the  main  effect  - pilot  workload  by  FOV 
pair. 

The  significant  result  of  the  subjective  workload  analysis  is 
o o 

that  the  6 and  1.5  FOV  pair  create  the  least  pilot  workload  as 
rated  by  pilots. 

4.2.3  Pilot  Effectiveness  Assessment 

As  in  the  wTorkload  analysis,  the  F-test  and  Friedman  test 
determined  one  effect  due  to  the  independent  variables.  Again,  the 
FOV  pair  variable  w7as  the  single  significant  influencing  factor  on 
pilot  effectiveness  as  rated  by  the  pilots.  The  mean  of  the  pilot 
effectiveness  ratings  for  each  FOV  pair  is  depicted  in  Figure 

4.2- 3.  As  shown,  the  6°  and  1.5°,  6°  and  0.86°,  and  3°  and  0.86° 

FOV  pairs  were  rated  by  the  pilots  in  a descending  order  of  effec- 
tiveness, respectively . The  3 and  0.86  FOV  pair  was  obviouslv 
lowest  in  effectiveness.  The  difference  between  the  6 and  1.5 
and  the  6 and  0.86  FOV  pairs,  although  larger  than  in  the  work- 
load analysis,  was  not  as  obvious;  howTever,  the  T-test  and  Fried- 
man tests  again  determined  a significant  difference  between  the 
pilot  ratings  for  each  FOV  pair.  The  detained  results  of  the  F- 
test,  T-test,  and  Friedman  tests  for  the  effectiveness  are  contained 
in  Appendix  B. 

The  F-test  determined  one  effect  due  to  the  interaction  be- 
tween independent  variables.  This  effect  was  due  to  the  inter- 
action of  FOV  pair  by  task.  The  graphical  analysis  of  this  inter- 
action is  depicted  in  Figure  4.2-4.  As  in  the  workload  analysis, 
the  source  of  the  effect  was  due  to  changing  trends  in  the  order 
of  pilot  FOV  pair  ratings  by  task.  This  is  indicated  in  Figure 

4.2- 4  by  the  crossover  between  data  points.  However,  the  basic 
trend  of  rating  the  6 and  1.5°,  the  6°  and  0.86°,  and  the  3°  and 
0.86  FOV  pairs  in  a descending  order  of  relative  effectiveness 
for  each  task  did  not  change.  Therefore,  the  interaction  between 
FOV  pair  and  task  variables  had  no  confounding  effect  on  the  main 
effect  - effectiveness  by  FOV  pair. 

The  significant  result  of  the  subjective  effectiveness  analysis 
is  that  the  6 and  1.5  FOV  pair  provide  the  most  effective  system 
as  rated  by  pilots. 


Figure  A. 2-3  Pilot  Effectiveness  Ratings  for  Each  FOV  Pair 


4.3  Objective  Data  Analysis 


The  analysis  of  workload  measures,  effectiveness  measures, 
acquisition  and  weapons  delivery  ranges,  and  narrow  FOV  employment 
was  conducted  on  the  data  retrieved  from  each  experiment. 

4.3.1  LGB/CCRP  Objective  Results 

Results  of  the  trend  analysis  conducted  on  workload  measure- 
ments are  shown  in  Table' 4.3-1.  The  analysis  determined  that  the 
6°  and  1.5  FOV  pair  created  the  lowest  workload  in  the  LGB/CCRP 
experiment.  Data  from  which  the  results  were  drawn  are  contained 
in  Appendix  B . 

A summary  of  successful  runs  is  contained  in  Table  4.3-2, 
which  shows : 

1.  The  number  of  runs  in  which  a successful  target  acquisition 
was  made  out  of  the  number  of  valid  runs. 

2.  The  number  of  runs  in  which  a successful  weapons  delivery 
was  made  out  of  the  number  of  runs  in  which  a successful 
target  acquisition  was  accomplished. 

3.  The  number  of  runs  in  which  a successful  weapons  delivery 
was  made  out  of  the  number  of  runs  in  which  a successful 
target  acquisition  was  accomplished  adjusted  for  non-pod- 
related  weapons  delivery  failures. 

The  adjustment  to  the  weapons  delivery  data  was  necessary  because 
the  number  of  non-pod-related  failures  did  not  balance  over  the  FOV 
pairs.  An  analysis  of  weapon  delivery  failures  for  each  FOV  pair 
on  which  the  adjustment  was  based  is  provided  in  Appendix  B. 

The  Friedman  test  on  the  target  acquisition  and  weapons 
delivery  success/failure  data  found  no  significant  effects  due  to 
FOV  pair  or  pop-up/REO  activation  range.  Appendix  B contains  the 
detailed  results  of  these  analyses. 

The  results  of  the  target  acquisition  and  weapons  delivery 
range  analyses  are  shown  in  Tables  4.3-3  and  4.3-4.  Because  of 
small  differences  between  the  mean  performances  and  large  standard 
deviations  in  these  data,  no  significant  results  could  be  deduced 
from  these  analyses.  However,  it  is  interesting  to  note  that, 
employing  the  6°  and  1.5  FOV  pair,  the  pilots  had  the  longest 
target  acquisition  range  for  all  REO  activation  ranges  and  the 
longest  weapons  delivery  ranges  for  the  6-n.mi  and  3-n.mi  REO 
activation  ranges  based  on  the  mean  performance  levels. 
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The  narrow-FOV  analysis  for  the  LGB/CCRP  experiment  is  shown 
in  Table  4.3-5.  The  frequency  of  narrow-FOV  utilization  was 
approximately  the  same  in  the  FOV  pairs  employing  the  6 vibe  FOV 
and  almost  non-existant  for  the  FOV  pair  employing  the  3 wide  FOV 
Significant  narrow-FOV  usage  for  target  acquisition  was  confined 
to  the  1.5°  FOV. 
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Pilots  always  began  run  in  wide  FOV 


4.3.2  LG  Maverick/LGM  Objective  Results 

Results  of  the  trend  analysis  conducted  on  workload  measures 
areoshown  in  Table  4.3-6.  The  analysis  determined  that  the  6°  and 
1.5  FOV  pair  created  the  lowest  workload  in  the  LG  Maverick/LGM 
experiment.  Data  from  which  the  results  were  drawn  are  contained 
in  Appendix  B. 

A summary  of  successful  runs  is  contained  in  Table  4.3-7.  The 
weapons  delivery  success  rates  were  again  adjusted  to  remove  the 
non-pod-related  failures  due  to  an  uneven  distribution  of  those 
failures  over  the  FOV  pairs.  An  analysis  of  the  weapon  delivery 
failures  on  which  the  adjustment  was  based  is  provided  in  Appendix 
B. 

The  results  of  the  target  acquisition  and  weapons  delivery 
range  analysis  are  shown  in  Tables  4.3-8  and  4.3-9.  Again,  the  re- 
latively small  difference  between  the  mean  performance  levels  and 
the  large  standard  deviations  in  the  target  acquisition  data  made 
it  impossible  to  deduce  any  significant  results  from  the  analysis . 
However,  it  is  interesting  to  note  that  the  pilots  had  the  longest 
target  acquisition  range  and  over  all  REO  activation  ranges  while 
employing  the  6 and  1.5  and  the  3°  and  0.86°  FOV  pairs.  The 
weapon  release  analysis  determined  a significant  result.  For  the 
two  longest  REO  activation  ranges,  REO  activation  at  run  initiation 
(ARI)  and  at  6 n.mi,  the  pilots  launched  the  Maverick  at  signifi- 
cantly longer  ranges  while  employing  the  6°  and  1.5°  and  the  3°  and 
0.86°  FOV  pairs. 

The  narrow-FOV  employment  analysis  for  the  LG  Maverick/LGM 
is  shown  in  Table  4.3-10.  The  frequency  of  narrow-FOV  utilization 
was  approximately  the  same  in  the  FOV  pairs  employing  the  6U  wide 
FOV.  In  the  3°  and  0.86  FOV  pair,  the  frequency  of  narrow-FOV 
utilization  was  high  (86%  of  the  runs)  for  the  longest  REO  activa- 
tion range  but  decreased  to  37.5%  for  the  intermediate  range  and 
was  non-existent  for  the  short  REO  activation  range.  Narrow-FOV 
usage  for  target  acquisition  was  significantly  higher  in  the  1.5° 
FOV  over  all  REO  activation  ranges. 
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4.3.3  LWLGM/LGMDT  Objective  Results 

Results  of  the  trend  analysis  conducted  on  workload  measures 
are  contained  in  Table  4.3-11.  At  one  pop-up  range  point  for  the 
6 and  1.5  FOV  pair  there  was  insufficient  successful  runs  to  make 
a validQworkload  determination.  The  analysis  shows  both  the  6 
and  1.5  and  the  6 and  0.86  FOV  pairs  to  be  lowest  in  workload, 
with  no  significant  difference  existing  between  the  two  for  those 
data  points  with  sufficient  data  to  make  a determination.  Data 
from  which  the  results  were  drawn  are  contained  in  Appendix  B. 

A summary  of  successful  runs  is  contained  in  Table  4.3-12. 

The  weapon  delivery  successes  have  been  adjusted  to  remove  the  non- 
pod-related failures.  A comparatively  high  target-acquisition 
failure  rate  existed  for  all  FOV  pairs  in  this  experiment.  A tar- 
get-acquisition failure  analysis  was  performed  to  determine  the 
cause  of  the  high  failure  rate.  The  analysis  determined  that  61  of 
the  78  failures  to  acquire  the  target  were  due  to  pilots  not  locat- 
ing the  target  area  visually  on  the  wind  screen  display  with  suf- 
ficient time  to  locate  the  targets  and  to  lock  the  pod  on.  The 
results  of  the  failure  analysis  are  contained  in  Table  4.3-13.  The 
high  target-acquisition  failure  rates  were  therefore  not  related 
to  the  EO  pod.  Instead,  the  failures  were  related  to  the  pilot's 
inability  to  employ  the  windscreen  display  to  navigate  and  locate 
the  target  area  heads-up.  Since  the  target  acquisition  success/ 
failure  data  did  not  relate  to  the  EO  pod,  this  measure  was  not 
analyzed  to  determine  system  effectiveness. 

The  Friedman  test  on  the  weapons  delivery  success/ failure  data 
found  no  significant  effects  due  to  FOV  pair  or  pop-up  range.  The 
detailed  results  of  these  analysis  are  given  in  Appendix  B. 

The  target  acquisition  range  data  did  not  reflect  the  pilots' 
capability  to  employ  the  pod  to  search  and  acquire  targets.  In- 
stead, it  indicated  where  the  pilots  located  the  target  area 
visually,  head-up  on  the  windscreen  display.  Since  weapon  delivery 
range  was  dependent  on  target  acquisition  range,  the  weapon  delivery 
range  data  did  not  reflect  the  pilot's  ability  to  employ  the  EO  pod 
to  deliver  a weapon.  Therefore,  the  range  data  analyses  contained 
in  Appendix  B for  the  LWLGM/LGMDT  are  not  representative  of  the 
pilots'  capability  to  employ  the  EO  pod  to  acquire  a target  or 
deliver  a missile. 

The  narrow-FOV  employment  analysis  for  the  LWLGM/LGMDT  experi- 
ment is  shown  in  Table  4.3-14.  The  narrow  FOVs  were  used  signifi- 
cantly more  often  in  the  FOV  pairs  employing  the  6°  wide  FOV. 

Narrow- F0VQusage  for  target  acquisition  was  significantly  higher 
in  the  1.5  narrow-FOV  over  all  pop-up  range  points. 
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Table  4.3-13  TARGET  ACQUISITION  FAILURE  ANALYSIS  RESULTS 
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Late  Acquisitions 
Switchology  Problem 


4.4  Field-of-Regard  Results 


The  seeker  field- of-regard  requirements  were  evaluated  through 
an  analysis  of  pod- sensor  line-of-sight  warning  and  obscuration 
data.  Pod-sensor  line-of-sight  warnings  and  obscurations  occurred 
predominantly  in  the  LGB/CCRP  experiment.  As  an  example  of  the 
dominance,  90  obscurations  occurred  in  the  LGB/CCRP  experiment,  12 
in  the  LG  Maverick/LGM  experiment,  and  none  in  the  LWLGM/LGMDT 
experiment.  Therefore,  only  the  data  obtained  from  the  LGB/CCRP 
experiment  were  analyzed; 

4.4.1  Pod  Sensor  Line-of-Sight  Warning  Results 

A total  of  255  pod- sensor  line-of-sight  warnings  occurred  in 
116  of  162  valid  runs.  Figure  4.4-1  shows  the  warning  onset 
implemented  for  the  study  and  the  number  of  warning  occurences  by 
10-degree  azimuth  increments.  Of  the  255  warnings,  128  were  due  to 
aircraft  or  stores  masking  and  127  were  due  to  pod- sensor  gimbal 
limits.  Of  the  128  warnings  due  to  aircraft  or  stores  masking, 

79  (62%)  occurred  between  -10  and  -50  degrees  of  azimuth.  An 
analysis  of  the  aircraft  attitude  at  these  warnings  determined  that 
the  warnings  were  due  to  excessive  left  roll  induced  by  the  pilots 
immediately  following  bomb  release  while  the  aircraft  was  pitched 
up  greater  than  10  degrees  for  a toss  maneuver.  Roll  to  the  right 
under  the  same  conditions  did  not  create  the  same  effect  because 
the  greater  f ield-of-regard  with  the  pod  mounted  on  the  right  side 
of  the  aircraft  inlet. 

4.4.2  Pod  Sensor  Line-of-Sight  Obscuration  Results 

A total  of  90  pod- sensor  line-of-sight  obscurations  occurred 
out  of  the  255  pod- sensor  line-of-sight  warnings.  The  90  obscura- 
tions occurred  in  73  runs.  Figure  4.4-2  shows  the  obscuration 
limits  implemented  for  the  study  and  the  number  of  obscuration 
occurrences  by  10-degree  increments.  Of  the  90  obscurations,  41 
(45%)  were  due  to  pod- seeker  gimbal  limits  and  49  (55%)  were  due 
to  aircraft  or  stores  masking.  Again,  the  major  portion  (70%)  of 
the  obscurations  due  to  aircraft  and  stores  masking  occurred 
between  -10  and  -50  degrees  of  azimuth. 

4.4.3  Pod  Gimbal  Limit  Analysis  and  Results 


The  pod- sensor  gimbal  angles  and  aircraft  ground  track  from 
weapon  release  to  weapon  impact  for  a typical  LGB  delivery  in 
which  the  simulated  pod- sensor  gimbal  limits  were  reached  are 
shown  in  Figure  4.4-3.  Of  specific  interest  during  the  delivery 
are  the  aircraft  ground  track  and  pod- sensor  gimbal  angles  from  15 
seconds  to  20  seconds  after  bomb  release.  During  this  period  the 
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Figure  A. 4-1  Wnrning  Onset  and  Number  of  Warnings  by  10°  Increments 
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the  aircraft's  longitudinal  axis  (tail  of  the  aircraft)  is  passing 
through  the  target,  which  is  at  the  origin  of  the  X and  Y axes. 

When  this  occurs,  the  pod  gimbal  translation  is  almost  totally  in 
the  elevation  axis,  as  indicated  by  the  vertical  movement  of  the 
line  describing  the  pod  gimbal  angles. 

Because  of  this  phenomenon,  the  simulated  pod- sensor  gimbal 
limits  were  defined  between  150  and  180  degrees  of  azimuth  at  zero 
elevation  to  detect  the  azimuth  angle  at  which  the  pod  gimbal  inter 
.sected  the  zero  elevation  line  (azimuth  axis)  . The  azimuth  angle 
at  the  zero  elevation  line  provided  an  approximation  of  the  pod- 
seeker  gimbal  back- look  requirement  to  contain  the  excursion.  This 
is  shown  in  Figure  4.4-3,  where  pod  gimbal  back- look  limits  of  150 
and  162  degrees  are  shown  on  the  right-hand  side  of  the  azimuth 
axis  by  dashed  lines.  Note  that  the  line  describing  the  pod  gimbal 
translation  clearly  violates  the  150-degree  back-look  limit  and 
that  the  162-aegree  back-look  limit  is  not  violated  since  the  line 
is  running  tangent  to  the  limit.  Also  note  that  the  line  describ- 
ing the  pod  gimbal  translation  crosses  the  zero  elevation  line 
(azimuth  axis)  at  approximately  162  degrees. 

The  point  at  which  the  line  describing  the  pod  gimbal  trans- 
lation begins  its  vertical  climb  and  crosses  the  azimuth  axis  is 
dependent  on  aircraft  altitude  above  the  target  and  aircraft  pitch 
and  roll  in  the  turn.  In  the  LGB/CCRP  experiment  the  point  at  whic 
the  line  began  its  vertical  climb,  thus  creating  pod  gimbal  limit 
warnings  and  limit  broaches,  was  almost  totally  dependent  on  air- 
craft altitude  above  the  target.  This  was  primarily  due  to  the 
canned  scenario  employed  in  the  study.  If  the  pilot  had  sufficient 
altitude  vrhen  a gimbal  limit  warning  occurred  he  could  duck  under 
the  gimbal  limit  by  rolling  the  aircraft  level  and  pushing  the 
nose  down. 

Three  techniques  for  measuring  the  required  pod-  sensor  gimbal 
limits  were  considered:  (1)  defining  gimbal  look-back  limits  and 
measuring  the  frequency  that  each  limit  was  used  during  the  run, 

(2)  defining  gimbal  look-back  limits  and  measuring  the  frequency 
that  each  limit  was  used  when  the  pilots  reached  a simulated  pod 
gimbal  limit  warning,  and  (3)  defining  gimbal  look-back  limits  and 
measuring  the  frequency  each  limit  was  used  when  the  simulated 
gimbal  limits  were  reached.  The  first  technique  was  not  used  since 
the  pilots  could  have  reached  a gimbal  limit  defined  for  analysis 
without  warning.  In  81  instances  of  127  pod  gimbal  limit  warnings, 
the  pilot  had  sufficient  altitude  and  time  available  to  avoid  simu- 
lated gimbal  limit  and  possibly  the  gimbal  limits  defined  for 
analysis,  therefore,  the  second  technique  was  not  appropriate. 

The  simulated  gimbal  limit  was  reached  because  the  pilot  was  either 
distracted  by  workload  and  did  not  react  quickly  enough  to  avoid 
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Che  simulated  gimbal  limit  or  did  not  have  sufficient  altitude  to 
escape  the  gimbal  limit.  In  either  case  the  pilot  would  not  have 
avoided  gimbal  limits  defined  for  analysis.  Therefore,  the  third 
technique  was  chosen  as  the  method  for  analvsis. 

The  pod  sensor  gimbal  limits  for  analysis  were  defined  in 
10-cegree  increments  from  150  to  180  degrees  of  azimuth.  The  fre- 
quency that  each  limit  was  used  when  the  simulated  gimbal  limits 
were  reached  is  graphically  portrayed  in  Figure  4.4-4. 

There  were  100  successful  runs  in  the  LGB/CCRP  experiment. 
Twelve  runs  were  scored  unsuccessful  because  of  a pod-sensor  line- 
cf-sight  obscuration  which  lasted  more  than  1 second.  Eight 
obscurations  were  due  to  pod  sensor  gimbal  limits,  and  four  ob- 
scurations were  due  to  aircraft/s  teres  masking.  Since  the  1- 
second  criteria  for  determining  run  success  or  failure  was  arbitrary 
for  pod  sensor  gimbal  limits,  a further  analysis  of  the  successful 
runs,  including  the  eight  failures  due  to  pod-sensor  gimbal  limits, 
was  conducted.  For  the  analysis,  pod-sensor  back-look  angles  of 
150,  150,  170,  and  180  degrees  were  defined,  and  the  percentage 
of  the  108  runs  in  which  the  gimbal  limit  had  no  factor  (was  not 
reached)  was  determined.  The  results  of  this  analysis  are  dis- 
played in  Figure  4.4-4  to  provide  a better  picture  of  pod  sensor 
gimbal  limits  in  the  context  of  run  success. 

The  frequency  analysis  of  gimbal  limit  usage  determined  that 
the  highest  frequency  occurred  between  pod-sensor  gimbal  back-look 
angles  of  160  and  170  degrees.  The  analysis  of  successful  runs 
determined  that  from  75%  to  95%  of  the  successful  runs  did  not 
have  obscurations  due  to  gimbal  limits  when  the  gimbal  limits  were 
defined  between  160  and  170  degrees. 

4.5  Summary  of  Results 

4.5.1  Subjective  Results 

The  analysis  of  pilot  ratings  employing  both  a parametric  F- 
test  and  a non-parametric  Friedman  test  determined  that  the  6°  and 
1.5  FOV  pair  provided  the  least  pilot  workload  and  the  most  system 
effectiveness . 

4.5.2  Objective  Results 


The  trend  analysis  of  workload  measures  from  the  three  experi- 
ments determined  that  the  6 and  1.5  FOV  pair  provided  the  lowest 
workload.  In  the  LWLGM/LGMDT  experiment,  no  significant  difference 
existed  between  the  6°  and  1.5°  and  the  6°  and  0.86°  FOV  pairs.  In 
the  two  other  experiments,  a significant  difference  occurred  be- 
tween each  FOV  pair. 
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Figure  4.4-4  Pod  Sensor  Gimbal  Limit  Analyses 
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The  analysis  of  effectiveness  measures  did  not  determine  any 
significant  difference  between  FOV  pairs. 

Because  of  small  differences  between  the  mean  performance 
levels  and  because  of  large  standard  deviations  in  the  data,  only 
one  significant  result  could  be  deduced  from  the  target-acquisition 
and  weapon-release  range  measures.  The  result  occurred  in  the  LG 
Maverick/LGM  exgeriment  where  pilots  employing  the  6°  and  1.5U  and 
the  3U  and  0.86  FOV  pairs  obtained  significantly  longer  weapons 
release  ranges  than  when  employing  the  6 and  0.86u  FOV  pair. 
However,  the  trends  in  these  data  supported  the  subjective  results 
in  all  instances. 

The  narrow-FOV  utilization  analysis  determined  that  the  narrow 
FOV  was  employed  significantly  more  often  in  the  FOV  pairs  having 
the  6U  wide  FOV.  Narrow-FOV  usage  for  target  acquisition  was 
significantly  higher  in  the  1.5U  narrow  FOV  over  all  experiments. 
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section  5 


CONCLUSIONS 


5.1  Individual  Fields-of-View 

Conclusions  concerning  each  individual  f ield-of- view  are  based 
on  pilot  remarks  and  observations,  pilot  f ield-of- view  pair  ratings, 
and  objective  experimental  measures.  These  conclusions  are  dis- 
cussed below. 


5.1.1  6-Degree  Wide  FOV 

/Then  employed  for  target  search,  the  6°  FOV  provided  excellent 
capability  at  all  ranges  against  all  targets.  When  emgloyea  for 
target  verification  against  large  fixed  targets,  the  6 FOV  was 
good  at  medium  to  close  ranges;  however,  against  small  mobile-type 
targets,  it  was  limited  to  close  ranges.  When  employed  for  target 
lock-on  against  large  targets,  the  6 FOV  was  good  at  all  ranges; 
however,  against  small  targets  it  was  limited  to  medium  to  close 
ranges . 


5.1.2  3-Degree  Wide  FOV 


The  3 FOV  was  too  narrow  for  target  search  operations  at 
attack  ranges  simulated  in  the  study.  However,  for  target  verifi- 


cation and  lock-on  operations,  the  3 
at  all  ranges  against  all  targets. 


FOV  provided  a good  capability 


5.1.3  1.5-Degree  Narrow  FOV 

The  1.5°  FOV  was  too  narrow  for  target  search  Operations  at 
all  ranges.  The  1.5  FOV  provided  an  excellent  capability  for 
target  verification  at  all  ranges.  Target  lock-on  employing  the 
1.5  FOV  was  good  at  all  ranges  against  small  targets;  however, 
it  was  limited  to  long  and  medium  ranges  for  large  fixed  targets . 

5.1.4  O.S6-Degree  Narrow  FOV 

The  0.86°  FOV  with  degraded  video  clarity  was  too  narrow  for 
the  attack  ranges  simulated.  Although  the  pilots  were  briefed  or. 
the  degraded  video  and  shown  a video  tape  of  an  EO  Pod  flight 
demonstration  containing  profiles  employing  1°  and  u°  FOVs 
with  excellent  video  quality  prior  to  the  experiments,  some 
bias  did  occur  as  a result  of  the  poor  video  quality.  The  amount 
of  bias  introduced  by  the  degraded  video  in  this  FOV  on  the  subjec- 
tive pilot  ratings  and  objective  performance  levels  was  impossible 
to  determine. 
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5.2  Field-c 


f-View  Pairs 

From  analysis  of  the  primary  data  source,  the  subjective 
pilot  f ield-of-view  ratings,  it  was  determined  that  the  6°  anc 
1.5°  FOV  pair  provided  the  least  pilot  workload  and  the  most 
system  effectiveness.  The  significant  results  from  the  analysis 
of  the  secondary  data  source,  objective  pilot  performance  measures, 
supported  the  primary  data  source.  Therefore,  for  the  simulated 
attack  scenario,  which  included: 

o Low-altitude  (500  feet)  , high-speed  (540  KTAS)  ingress 

o Pop-up  for  target  acquisition  and  weapons  delivery 

o Visibility  restrictions  ranging  between  6 and  3 n.mi 

o A weather  ceiling  of  2500  feet 

and  for  the  simulator  limitations,  which  included: 

o Degraded  video  clarity  in  the  0.86°  FOV 

o A 50-degree  (measured  diagonally)  forward  visibility 
provided  by  a 500- line,  black  and  white  video  monitor 

o A terrain  board  measuring  80  ft  by  40  ft  that,  at 

1200-to-l  scale,  provided  a working  area  of  14  n.mi  by 
7 n.mi 

the  6°  and  1.5°  FOV  pair  is  best. 

5.3  Pod  Seeker  Field-of-Regard 

The  results  of  the  pod-sensor  line  of  sight  warning  and  pod- 
sensor line-of-sight  obscuration  analyses  indicated  that  the  pilots 
responded  well  to  the  warning  tone.  Of  the  255  warnings,  only  90 
resulted  in  pod-sensor  line-of-sight  obscurations  or  gimbal  limits. 
These  results  indicate  the  need  for  a warning  implementation  in 
aircraft  carrying  an  advanced  EO  pod. 

The  results  of  the  pod-sensor  gimbal  limit  analysis  indicate 
a gimbal  back-look  capability  defined  between  160  and  170  degrees 
is  required  for  the  delivery  of  laser  guided  bombs . 


67 


REFERENCES 


1 . F-16  Advanced  Electro-Optical  Pod  Field-of-View  Study 
Flight  Test  Plan,  FZM-6781,  General  Dynamics,  Fort  Worth, 
Texas,  1978. 

2.  Cochran,  N7.  G.  and  Cox,  G.  M.  , Experimental  Designs, 
Chapter  3,  New  York:  John  Wiley  and  Sons,  Incorporated, 
1957. 

3.  Winer,  B.  J.,  Statistical  Principles  in  Experimental 
Design , Chapter  4 and  Appendix  A,  New  York:  McGraw-Hill 
Book  Company,  Incorporated,  1962. 

4.  Conover,  W.  J.,  Practical  Nonoarametric  Statistics , 
d.  274,  New  York:  John  Wiley  and  Sons,  Incorporated, 

*1971. 


68 


r 


APPENDIX  A 

F-TEST,  T-TEST,  AND  FRIEDMAN  TEST 
FORMULAS 


A-l 


DETAILS  OF  F-TEST  APPLIED  TO  PILOT  WORKLOAD 

RATINGS 


■ 

SOURCE 

DEGREES 

OF 

FREEDOM 

ESTIMATED 

MEAN 

SQUARE 

F-TEST 

a 

Subjects  (A) 

5 

None 

2 

Mode  ( B) 

3 

2 2 

sa  r +9cr 

34  B AB 

© + © 

3 

Field  of  View 
(C) 

2 

54  rc  + 9 alc 

© • © 

D 

Task  (D) 

2 

54  rD  + 9%2D 

© + © 

5 

A X B 

15 

None 

6 

A X C 

10 

9trA2C 

None 

D 

A X D 

10 

9°iD 

None 

8 

B X C 

6 

© 

© 

9 

B X D 

6 

18  t 2 + 3^  2 

BD  ABD 

® * © 

10 

C X D 

4 

msm 

© * © 

11 

A X B X C 

30 

2 

3 °"abc 

None 

12 

A X B X D 

30 

3<tabd 

None 

13 

A X C X D 

20 

3 %2CB 

None 

14 

B X C X D 

12 

6 7bcd  + °"abcd 

© ♦ © 

15 

A X B X C X D 

60 

2 

^ABCD 

None 

DETAILS  OF  F-TEST  APPLIED  TO  PILOT  EFFECTIVENESS 

RATINGS 


SOURCE 

DEGREES 

OF 

FREEDOM 

Subjects  (A) 

5 

Mode  (B) 

3 

Field  of  View 
CC) 

2 

Task  (D) 

3 

A X B 

15 

A X C 

10 

A X D 

15 

B X C 

6 

B X D 

9 

C X D 

6 

A X B X C 

30 

A X B X D 

45 

A X C X D 

30 

B X C X D 

18 

A X B X C X D 

90 

TOTAL 

287 

ESTIMATED 

MEAN 

SQUARE 


8<T1 


+ 12  cr 


+ 16  0- 


72  TD  + 12°Nd 


F-TEST 


2 2 
24  t +4  <r 
4 CD  4 AC 


2 

^ABCD 


None 


© + © 


©*  © 


© + © 


None 


None 


None 


© + © 


© * © 


@ + © 


None 


None 


None 


© + © 


None 


-3 


T-TEST 


The  t 

1 


value  was  calculated  using  the  following  method. 


The  appropriate  standard  deviation  for  the  denor.ir 
of  the  t-test  was  estimated  by: 


ator 


Estimating  o~  from  the  'lean  Square  used  as  a 
nator  in  the  F-test. 


denomi 


b.  Estimating  ol  by: 


o 

ol  = 


in 


chere  n = Number  of  Observations 
.-iean 


m tr.e 


Estimating  o~ 


'1 


_ bv  : 
X2  ' 


2 

ol 

x,  - X, 


2 ol. 
x 


d.  The  denominator  was  then  determined  bv: 


S_  _ = 4 I ol 

X1  ' X2  \ X1  X2 

t was  then  calculated  by: 
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FRIEDMAN  TEST 


Test  equation  is  for  ranked  data  with  ties  between  rankings. 


k 


For  Subjective  Pilot  FOV  Pair  Ratings: 

n = Number  of  Raters 
k = Number  of  Conditions  Being  Rated 

X„  = Ranked  Rating  from  the  ith  - Rater  for  the  j th  - Condition 

For  Objective  Success /Failure  Data: 
n = Number  of  Subjects 

k = Number  of  Parameters  Over  Which  Subject  Performance  is 
Being  Ranked 

X..  = Ranked  Performance  Level  from  the  ic^‘  - Subject  for  the 
j t’n  _ Parameter 
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APPENDIX  B 

SUPPORTIVE  DATA  RESULTS 
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SUBJECTIVE  DATA  RESULTS 


F-Tesc  Results  on  Pilot  Rated  Workload 


SOURCE 

DEGREES 

OF 

FREEDOM 

SUM 

OF 

SQUARES 

MEAN 

SQUARE 

F 

VALUE 

PROBA- 

BILITY 

PILOT 

5 

41.81 

8.36 

MODE 

3 

7.59 

2.53 

0.43 

mm 

PILOT  X MODE 

15 

87.88 

5.86 

FOV 

2 

315.52  i 

157.76 

66.62 

0.01 

PILOT  X FOV 

10 

23.62 

2.37 

MODE  X FOV 

6 

10. SO 

1.80 

1.15 

N.S. 

PILOT  X MODE  X FOV 

30 

47.15 

1.57 

TASK 

2 

0.29 

0.15 

3.65 

IB 

PILOT  X TASK 

10 

22.35 

2.24 

MODE  X TASK 

6 

7.54 

1.26 

1.16 

■ZB 

PILOT  X MODE  X TASK 

30 

32.51 

1.08 

FOV  X TASK 

4 

7.36 

1.97 

6.20 

0.01 

PILOT  X FOV  X TASK 

20 

6.34 

0.32 

MODE  X FOV  X TASK 

12 

3.33 

0.28 

0.87 

N.S. 

PILOT  X MODE 

X FOV  X TASK 

60 

19.03 

0.32 

N.S.  --  No  significance  based  on  0.05  criterion 


Friedman  Test  Results 


Friedman  Test  Results 


SUBJECTIVE  DATA  RESULTS 
F-Test  Results  on  Pilot  Rated  Effectiveness 


SOURCE 

DEGREES 

OF 

FREEDOM 

SUM 

OF 

SQUARES 

MEAN 

SQUARE 

F 

VALUE 

PROBA- 

BILITY 

PILOT 

5 

140.02 

23.00 

MODE 

3 

29.30 

9.77 

1.30 

N.S. 

PILOT  X MODE 

15 

112.62 

7.51 

FOV 

2 

492.76 

246.33 

70.52 

0.01 

PILOT  X FOV 

10 

34.94 

3.49 

MODE  X FOV 

6 

8.15 

1.36 

0.65 

N.S. 

PILOT  X MODE  X FOV 

30 

63.01 

2.10 

TASK 

3 

4.34  ' 

1.45 

1.03 

N.S. 

PILOT  X TASK 

15 

20.98 

1.40 

MODE  X TASK 

9 

5.48 

0.61 

0.68 

N.S. 

PILOT  X MODE  X TASK 

45 

40.21 

0.89 

FOV  X TASK 

6 

52.30 

8.72 

5.22 

0.01 

PILOT  X FOV  X TASK 

30 

50.09 

1.67 

MODE  X FOV  X TASK 

18 

7.82 

0.43 

0.65 

N.S. 

PILOT  X MODE 

X FOV  X TASK 

90 

59.74 

0.66 

N.S.  --  No  significance  based  on  0.05  criterion 
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Target  Overflight  Failures  --  Pilots  were  briefed  that  target 
overflight  would  result  in  delivery  failure.  In  instances 
above  pilots  could  have  avoided  target  overflight  hut  did  not 


l.CK/CCRP 
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Friedman  Test  Results 


LGB/CCRP 
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LG  MAVERICK/LCM 


LG  MAVF.RICK/LCM 


Friedman  Test  Results 


Friedman  Test  Results 


sec  sec  I sec  II  sec  sec  I sec  sec  I sec  I sec 


LWLGM/LGMDT 


Missile  Delivered  Out  of  Off  1 - Tracker  Break  Lock 

Boresight  Limits 


Friedman  Test  Results 
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APPENDIX  C 

Simulation  Facility  Details 
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SIMULATION  FACILITY 


1.1  Terrain  Board 

The  terrain  model  was  80'  x 40'  with  a fiber  glass 
surface.  Three  foot  high  mirrors  ringed  the  terrain  model 
for  terrain  extension.  The  terrain  model  was  scaled  to 
1200:1  for  the  simulation.  Simulation  parameters  for 
1200 : 1 were : 


1200:1 


Parameter 

FT 

NM 

KM 

Slant  Range  to 

96000 

15.8 

29.3 

Lateral  to 

48000 

7.9 

14.6 

Altitude  to 

12000 

Ft  (Max) 

84 

Ft  (Min) 

Longitudinal  Velocity 

to 

12000 

Ft/Sec 

Vertical  Velocity 

to 

7200 

Ft/Sec 

Lateral  Velocity 

to 

4800 

Ft/Sec 

The  terrain  model  consisted  of  mountains,  plains,  har- 
bors, streams,  ridges,  forests,  dams,  tunnels,  highways, 
railroads,  bridges,  airports,  buildings,  docks,  etc. 

Topography  was  rolling  hills  modelled  after  West  Germany. 
Choice  of  materials  used  in  the  manufacture  of  the  terrain 
model  was  based  on  weather  and  sun  resistance  and,  at  the 
same  time,  a presentation  of  realistic  targets  to  the  sensors. 

1.2  Motion  Simulation 


The  3-D  terrain  model  was  mounted  on  30  trucks,  and  10 
central  trucks  with  compound  bearings  provided  lateral 
guidance  in  addition  to  vertical  guidance.  The  trucks 
allowed  translation  of  the  assembly  in  a longitudinal  direc- 
tion on  three  tracks.  The  characteristics  of  the  terrain 
model  longitudinal  drive  assembly  are  listed  below. 


Displacement 

Velocity 


Acceleration 

Positioning 


+ 80  ft  (probes  up) 

+ 78  ft  (probes  down) 
Accuracy  at  min  + 1.07o 
at  0.01  ft/sec 
Accuracy  at  max  + . 057« 
at  10  ft/sec 
+10.0  ft/sec 
Accuracy  +0.1  inch 
Repeatability  .02  inch  max 
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■ 


Small  signal  fre-  3.0  cps 

quency  response 

Weight  of  longitudinal  24,000  lbs.  static  and  rolling 
drive  system  friction  less  than  250  lbs. 

The  3-D  terrain  model  was  mechanized  to  simulate  the  longi- 
tudinal movement  of  the  aircraft  in  flight;  and,  therefore, 
accounted  for  one  degree  of  freedom. 

The  remaining  two  degrees  of  translational  freedom  to 
simulate  vertical  and  lateral  movement  of  the  aircraft  were 
provided  by  a lateral  carriage  and  a horizontal  beam.  The 
lateral  carriage  was  free  to  translate  laterally  and  was 
attached  to  a horizontal  beam  which  was  free  to  move  in  the 
vertical  direction  between  two  supporting  columns.  The 
operating  characteristics  of  the  lateral  and  vertical  drive 
systems  were: 

Lateral  and  Vertical  Drive  Characteristics 


Displacement 

Velocity 

Accuracy  at  minimum 
Accuracy  at  maximum 
Acceleration 
Positioning 
Accuracy 
Repeatability 
Small  Signal  Frequency 
Response 


Lateral 
38  ft 

+ 1.0%  at  0.004 
.05%  at  4.0  ft/s 
4 ft/s2 

+ . 02  inch 
.005  in.  max 

3.0  cps 


Vertical 

25  ft  7 in. 

+ 1.0%  at  0.00 
.05%,  at  6.0  ft/s^ 

6.0  ft/s2 

+ . 02  inch 
.005  in.  max 

3.0  cps 


1.3  Video  System 
1.3.1  Optical  Probes 

Two  optical  probes  were  employed  to  provide  wide  FOV 
imagery  for  the  windscreen  display  and  narrow  FOV  imagery 
for  the  simulation  of  the  EO  pod.  The  wide  FOV  probe  was  a 
Schemimpflug  corrected  probe  which  had  essentially  an 
infinite  depth  of  field,  allowing  low  level  operation  over 
the  terrain  board.  The  specifications  of  this  probe  were: 


1. 

Field  of  View 

50°  circular 

2. 

Minimum  altitude 

63mm  a 250 

ft 

(1200:1) 

3. 

Near  focus 

25mm  a 100 

ft 

(1200:1) 

1 


L 
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2.3  arc  min  at  257=  MTV 


L. 


4.  Resolution 

5.  Servo  Performance 

Displacement 
Roll  Continuous 

Pitch  +25°,  -90° 

Yaw  Continuous 


Velocity 
360  /sec 
100°/sec 
360°/sec 


The  narrow  FOV  probe  was  used  to  generate  the  imagery 
for  the  EO  probe  simulation.  It  was  also  a Schemimpflug 
corrected  probe  which  had  essentially  an  infinite  depth  of 
field,  allowing  low  level  operation  over  the  terrain  board. 
The  specifications  of  this  probe  were: 


1. 


2. 

3. 

4. 

5. 


Fields -of -View 


Minimum  altitude 
Near  focus 
Resolution 
Servo  Performance 


6°  circular 

4U  circular 

1.5  circular 

0.86°  circular 

12  ran  a 50  ft  (1200:1) 

30  mm  ~ 120  ft  (1200:1) 

17  arc  sec  at  257=  MTV 


Roll 

Pitch 

Yaw 


Displacement 
Continuous 
+25°,  -90° 
Continuous 


Velocitv 
360°/sec 
100°/sec 
360°/ sec 


The  longitudinal  separation  between  entrance  pupils 
of  the  two  probes  was  12  inches  (1200  scale  feet) . The 
altitude  separation  between  the  entrance  pupils  of  the  two 
probes  was  2 inches  (200  scale  feet).  The  resulting 
parallex  effects  were  compensated  with  software  on  the 
hybrid  computer. 


1.3.2  Video  System 

Two  1200  lines,  60  MHz  high  resolution  TV  systems  were 
used  with  the  optical  probes  to  generate  the  video  for  the 
windscreen  display  and  the  radar/electro-optical  (REO)  dis- 
play. Both  systems  had  a variable  line  rate  and  bandwidth 
so  that  a 512  line  signal  could  be  generated  for  the  study. 

In  conjunction  with  the  optical  probes,  the  TV  systems 
yielded  resolutions  of  4.4  arc  minutes  for  the  windscreen 
probe  at  the  50-degree  field-of-view  and  17  arc  seconds  for 
the  REO  display  probe  at  the  1.5-degree  field-of-view.  The 
TV  systems  had  the  capability  to  be  underscanned  down  to  a 
maximum  ratio  of  2:1.  The  3-degree  field-of-view  employed 
in  the  study  was  created  by  underscanning  the  4-degree  field- 
of-view. 
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1.4  Analog/Digital  Facility 


The  computational  elements  of  the  facility  are  listed 
below. 

Digital  Computer 
Sigma  5 - Triple  CPU 


Memory  Size 
Word  Length 
Memory  Cycle  Time 
Arithmetic 


160K  words 
32  bits 
1.0  fjLS 
Fixed  point 
point 


and  floating 


Analog  Computers 


231R-V' s 

Number  of  Consoles  6 

Total  number  of  amplifiers  1496 

Quarter-square  multipliers  276 

Resolvers  30 

Potentiometers  - Servo  Set  900 

Potentiometers  - Hand  Set  140 

Function  generators  120 


Hybrid  Interface 


Digital  Computer/231R-V 


Multiplying  Digital  to 

56 

Analog  Converters 

Analog  to  Digital  Con- 

48 

verter  Channels 

12  BIT  DAC 

64 

Peripheral  Devices  - Sigma  5 


2 Card  Readers 
4 Mag  Tapes 

2 Line  Printers 

2 Fixed  Head  Disk  Memories 
1 Disk  Pack 


400  & 1500  car/min 

75  inches  per  second 

800  bits/inch 

800  Sc  1000  lines/min 

132  char/line 

6.0  Mbytes 

48  Mbytes 


C-5 


